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FAST EDGE-ORIENTED IMAGE INTERPOLATION ALGORITHM 

BACKGROUND OF THE INVENTION 

Field of Invention 

[0001] The present invention relates to image scaling. More particularly, the 
present invention relates to image interpolating method to zoom a digital image. 

Description of Related Art 

[0002] Multimedia communication is becoming important in modern networks. 
Human can talk face to face and exchange the information each other by Internet and 
wireless communication. However, the digital image and video sequence have larger 
size of data, and network bandwidth is limited, thus the quality of multimedia commu- 
nication decreases. If the low-resolution video sequences are coded in the encoder, and 
the low-resolution video sequences are enlarged to high-resolution ones using interpo- 
lation techniques in the decoder, the bandwidth will be saved. 

[0003] Briefly, the image interpolation is about image enlargement. Some 
common interpolation algorithms are proposed, which include zero-order interpolation, 
linear interpolation and cubic convolution interpolation, ... etc. To solve artifact ef- 
fects, many algorithms are proposed, which interpolate images using pixels feature. 
However, the methods of determining pixel feature need higher computational com- 
plexity, and do not achieve real-time (30 frames/sec) image enlargement in video se- 
quence. 
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[0004] To solve artifact effects for image interpolation, many algorithms are 
proposed. The algorithm can be based on spatial domain direction. The interpolated 
values along various directions are combined using directional weights, which depend 
on the variation in the direction. The algorithms also include convolution-based, 
5 wherein the interpolated pixels are classified to two decimations. The different decima- 
tions are interpolated using different filters, respectively. Also, a hybrid of convolution 
and median-based is conventionally used by splitting the interpolation into two direc- 
tional stages. 

[0005] Also and, a NEDI (new edge-directed interpolation) algorithm is a good 
10 algorithm, which produces better subjective quality for image enlargement. The basic 
idea in NEDI algorithm is first to estimate local covariance coefficient from a low- 
resolution image and then use these covariance estimates to adapt the interpolation at a 
higher resolution based on the geometric duality between the low-resolution covariance 
and the high-resolution covariance. The covariance-based interpolation algorithm has 
1 5 higher computational complexity. 

[0006] The high-resolution covariance is estimated from its low-resolution 
counterpart based on their intrinsic "geometric duality". Geometric duality refers to the 
correspondence between the high-resolution covariance and low-resolution covariance 
that couple the pair of pixels at the different resolutions but along the same orientation. 
20 FIG. 1 shows the geometric duality between the high-resolution covariance Rfci, r£ and 
the low-resolution covariance f k when interpolating the interlacing lattice 

Y2i+l,2j+l f orm ^2i,2j- Geometric duality facilitates the estimation of local covariance 
for 2-D signals without the necessity of explicitly estimating the edge orientation. 
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Similar geometric duality can also be observed in FIG. 2 when interpolating the inter- 
lacing lattice Yij(i+j = odd) from the lattice Yij(i+j = even). In fact, FIG. 1 and FIG. 2 



are isomorphic up to a scaling factor of 2 and a rotation factor of n I A. 

[0007] The low-resolution covariance r u , f k can be easily estimated from a local 
window of the low-resolution image using the classical covariance method 

R = J-C T C, ? = ~^C r y (1) 
M M 

where y = [y,~ m y k - -y M2 J 2 is the data vector containing the Mx M pixels inside the lo- 
cal window and C is a 4x data matrix whose &-th column vector is the four nearest 

neighbors ofyfc along the diagonal direction. The interpolated value of Y2i+l 9 2j+l can 
be obtained in the following 



k=0 1=0 



According to classical Wiener filtering theory, the optimal minimum mean squared error 
(MMSE) linear interpolation coefficients are given by 

a=/r'r (3) 

According to (1) and (3), a is obtained in the following. 

a = (c T cY(c T y) (4) 

[0008] The NEDI algorithm can be used to magnify the size of a grayscale im- 
age by any factor that is a power of two along each dimension. In the basic case where 
the magnification factor is just two, the resizing scheme consists of two steps: the first 
step is to interpolate the interlacing lattice Y2/+ 1, 2j+ 1 from the lattice Y2i t 2p the 
second step is to interpolate the other interlacing lattice Yjj(i+j = odd) from the lattice 
Yijii+j = even). 
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[0009] Even though many interpolation algorithms have been proposed, the cur- 
rent interpolation algorithms still consume a lot of computation time to zoom an image 
up. A fast interpolating algorithm is still under developing. 

SUMMARY OF THE INVENTION 
[0010] The invention provides an image interpolating method for zooming the 
low resolution pixels Yij of an image into the high resolution pixels Y2i t 2y The ed g e " 
orient is considered, so that the interpolation can be faster and image quality can remain 

at an acceptable level. 

[0011] The invention provides an image interpolating method, which comprises 
receiving the low resolution pixels Yij. Then, a homogenous area and an edge area of 
the image are determined according to pixel differences of the pixels J^i, 2j in compar- 
ing with a threshold. Then, the pixels Y2i t 2j belonging to the homogenous area are in- 
terpolated by a first interpolating algorithm, while the pixels Y2i 9 2j belonging to the 
edge area are interpolated by a second interpolating algorithm. 

[0012] In the foregoing interpolating method, the step of determining the homo- 
genous area and the edge area of the image uses three variables of 
AY] = \Y2iJj ~ Y 2 i+2p,2j+2q\ , P ,qe {(o,l),(l,o)}, 

A Y 2 = \Y2i+2,2j ~ Y 2i,2j+2\ > ™ d 

AY 3 = \Y 2it 2j -Y 2i +2,2j+2\ 
to determine whether the homogenous area or the edge area by a condition set of: 

if A Y\ < the threshold then 

the pixel Y2i+p t 2j+q is in the homogenous area 
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else 

the pixel Y2i+p t 2j+q is in the ed S e as one of edge pixels; 
if A Y2 < the threshold and A Y3 < the threshold then 
the pixel 72/+ 7,2/+ 1 ls m the homogenous area 
else if AY2< the threshold then 

the pixel Y2i+ l,2j+ 1 is xn ^ homogenous area 
else if A Y3 <the threshold then 

the pixel F2i+ 7,2/+ 7 * s in homogenous area 

else 

the pixel Y2i+ l,2j+ 1 is in the edge area as one of edge pixels. 

[0013] In the foregoing interpolating method, the second interpolating algorithm 
can include interpolating the pixels Y2i,2j alon 8 a direction having a minimum differen- 
ce in the neighboring pixels. 

[0014] In the foregoing image interpolating method, the neighboring pixels of 
one of the pixels Y2i 9 2j does not include the determined edge pixel. 

[0015] In the foregoing image interpolating method, when the minimum differ- 
ence diffmin is determined by taking a minimum of four differences of 

diffl = I Y 2i-l t 2j~ Y 2i+l,2j\> 

di ff2 = \ Y 2i-l 9 2j~l - Y 2i+l,2j+l\* 

dijf3 = \ Y 2i,2j-l ~ Y 2i,2j+l\* 311(1 

diff4 = \ Y 2i+l,2j-l " r 2i-1.2j+ll* 
wherein the differences including one of the edge pixels is skipped. 
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[0016] In the foregoing image interpolating method, the pixel Yq is obtained by 
calculating (Y 2i _] 2 j + Y 2i+1 2p 1 2 at a direction with the minimum pixel difference. 

[0017] It is to be understood that both the foregoing general description and the 
following detailed description are exemplary, and are intended to provide further ex- 
planation of the invention as claimed. 



BRIEF DESCRIPTION OF THE DRAWINGS 
[0018] The accompanying drawings are included to provide a further under- 
standing of the invention, and are incorporated in and constitute a part of this specifica- 
10 tion. The drawings illustrate embodiments of the invention and, together with the de- 
scription, serve to explain the principles of the invention. 

[0019] FIG. 1 is a drawing, showing the geometric duality between the high- 
resolution covariance Rkl r£ and the low-resolution covariance r u , f k when interpolat- 
ing the interlacing lattice Y2i+l t 2j+l form Y2i 9 2y 
1 5 [0020] FIG. 2 is a drawing, showing a similar geometric duality when interpo- 

lating the interlacing lattice Yij(i+j = odd) from the lattice Yij(i+j = even). 

[0021] FIG. 3 is a flow diagram, schematically illustrating the process of the im- 
age interpolating method, according to a preferred embodiment of the invention. 

[0022] FIG. 4 is a drawing, schematically illustrating the pixel difference in 
20 three directions, according to a preferred embodiment of the invention. 

[0023] FIG. 5 is a drawing, schematically illustrating the result of an image Lena 
after the first step of the interpolating algorithm, according to a preferred embodiment of 
the invention. 
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[0024] FIG. 6 is a drawing, schematically illustrating the neighboring pixels. 
[0025] FIG. 7 is a drawing, schematically illustrating the interpolation of the 
homogenous pixels. 

[0026] FIG. 8 is a drawing, schematically illustrating the interpolation of the 
edge pixels. 

[0027] FIG. 9 is a drawing, schematically illustrating the interpolation results by 
various algorithms for artificial image A. 

[0028] FIG. 10 is a drawing, schematically illustrating a subjective quality of the 
portion of Lena image. 

[0029] FIG. 1 1 is a drawing, schematically illustrating a subjective quality of the 
portion of Pepper image. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[0030] As previously mentioned, there are many conventional image interpola- 
tion algorithms having been proposed. However, the conventional method at least 
causes a calculation load. To solve the problem, a novel interpolation algorithm for im- 
age enlargement is introduced in the invention. The algorithm provides better subjective 
quality and lower computational complexity for image enlargement. 

[0031] To design the new algorithm, at least two goals, which are lower com- 
putational complexity and better subjective quality, have been considered. The novel 
interpolation algorithm of the invention can be at least applied, for example, to video 
sequences and videoconference efficiently. However, a still image can also be enlarged 
by the image interpolation algorithm of the invention. 
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[0032] The image interpolation method to interpolate image preferably bases on 
analyzing the local structure. The original images are segmented dynamically into two 
areas: homogenous area and edge area. The interpolated pixels in the different areas 
correspond to different interpolation algorithms, respectively. 

[0033] In the algorithm of the invention, the interpolated pixels are determined 
to belong to homogenous area or edge area using a threshold value. The threshold value 
for example is 10% of the full displaying range of the pixel value. For example, the full 
displaying range of gray level images are from 0 to 255, and the threshold is 25. The 
algorithm of the invention includes two steps 100 and 102, as shown in FIG. 3. Jn step 
104 of the first step 100, a definition is made for the differences on the horizontal, verti- 
cal and diagonal directions, respectively, in the 3x 3 window. Referring to FIG. 4, the 
differences on three directions one by one are determined. If the pixel difference is less 
than the threshold value, the pixel belongs to a homogenous area, wherein the homoge- 
nous pixel is interpolated using, for example, bilinear interpolation algorithm (step 106). 
If the pixel difference is bigger than the threshold value, it belongs to an edge area. Af- 
ter the first step 100, some remaining non-interpolated pixels then belong to the edge 
area. Second, the edge pixels are interpolated by the proposed algorithm of the inven- 
tion, which interpolates edge pixels using neighboring pixels information. The neigh- 
boring pixels contain original pixels and interpolated pixels in the first step 100. 

[0034] FIG. 5 shows the result of first step 100 of the invention for Lena image. 
Some non-interpolated pixels exist at the image edge. In the second step 102 of the in- 
vention, the edge pixels are interpolated using all neighboring pixels, which contain 
original pixels and interpolated pixels at first step. As shown in FIG. 6, the neighboring 
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pixels contain black-points, gray-points and spot-points. We know the smallest pixel 
difference imply the greatest relation between the pixels. Furthermore, the edge pixel is 
interpolated along the direction of the minimum difference. If all of the spot-points be- 
long to interpolated pixels at the first step 100, the minimum difference is found on four 
5 directions through the white-point (edge pixel). If any spot-points belong to edge pix- 
els, the minimum difference is found on the other directions excluding the edge pixels. 
In the following, the proposed algorithm is described in detail. 

[0035] Referring to FIG. 7, it is assumed that the X of low-resolution image with 
size Hx Wis enlarged to the Y of high-resolution image with size 2H x 2W. The 
10 Y2i,2j is zoomed from Xgj and it is determines about the homogenous pixels at 
Y 2i+l,2j> Y 2i,2j+1 and Y2i+l t 2j+l using pixel difference criterion. If these pixels are 
homogenous pixels, it is interpolated for these pixels using, for example, bilinear inter- 
polation algorithm. The pixel difference criterion is described as following: 
A Y l = | Y 2i 2 j ~ Y 2i+2p 2 j +2q | 
15 AY 2 = |Y 2i+2>2j - Y 2i , 2j+2 | 

A Y 3 = | Y 2i 2 j Y 2i+2 2 j +2 | 
if A Y, < threshold then 

Y 2 i+p,2j+q = (Y 2it2 j + Y 2i+2p 2 j +2q ) / 2 (5) 

else 

20 Y 2Rp,2j +q are edge pixel (6) 

where p.q <={(0,l),{l.0)} 

if A Y 2 < threshold and A Y 3 < threshold then 
AY^minlAY,, A Y 3 } 
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if AY min =AY 2 



Y2i+i,2j+l (Y 2 i+2,2j + ^2i,2j+2) I 2 (7) 

else 

Y 2 i+i,2j+i = (Yiuij + Y 2i+2 2 j +2 ) / 2 (8) 
else if A Y 2 < threshold then 

Y 2 i+i,2j+i = (Y 2 i +2 ,2j + Y 2i 2j+2 ) / 2 (9) 
else if A Y 3 < threshold then 

^2i+l,2j+l = (Y 2 i )2 j + Y 2i+2t 2j+2) / 2 (10) 

else 

Y 2i+i,2j+i is edge pixel. (1 1) 



The edge pixels are the non-interpolated pixels after the decision of pixel difference 
criterion and processed by following procedure. 

[0036] An Edge-Oriented Adaptive Interpolation for the edge pixels of the in- 
vention is described as an example. 

[0037] To find the minimum difference, two cases are described in the follow- 
ing. Case 1, all of the neighboring pixels are already interpolated at first step. The algo- 
rithm interpolates the edge pixels by the neighboring pixels of four directions. The 
minimum difference among four directions is evaluated first. The direction of the mini- 
mum difference represents that the edge pixel is oriented to this direction as shown in 
FIG. 8. The procedure of the minimum difference algorithm is shown as following: 
= I Y 2 , u . - Y 2i+ J 

diff 2 = |Y 2 , uj , - Y 2j+|>2j J 
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diff 4 = |Y 2i+I , 2j ,-Y 2M2j+I | 

diff m . n = min{diff R }, for k= 1-4 (12) 

[0038] In case 2, some of neighboring pixels are not interpolated at first step. In 
other words, some of neighboring pixels belong to the edge pixels, too. The minimum 
difference is found by using remaining neighboring pixels. The case 2 is explained in 
the following. Referring to FIG. 8, it is assumed that two pixels Y2i-l t 2j+l and 
Y 2i t 2j+1 are not interpolated at first step as an example, then the minimum difference is 
found by skipping these two directions. The procedure shows in the following. 

diffl = I Y 2i-l,2j ~ Y 2i+1>2 j| 

diff 2 = |Y 2i _, 2 j_, — Y 2i+1 2 j +1 | 

diff min -min{diff k }, fork=l-2 (13) 

[0039] From the above cases, we can find the orientation of the diff m i n . For ex- 
ample, if the diffmin is found as diff], Y 2i ,j 2j correlates closely with Y 2i +j 2j> and 
Y2i t 2j is interpolated using (14). 

Y2U2J = (Y 2i , l2 j + Y 2i+l,2j> /2 d4) 

[0040] Experiments are performed to verify the image interpolating method of 
the invention. It has been tested for comparing subjective and objective qualities of 
Zero-order, Bilinear, Bicubic, NEDI and proposed algorithms. Six gray-level images, 
Pepper, Milkdrop, Tiffany, Comtal, Jet, Lena and three color images, Pepper, Jet, Lena 
are tested. Our goals are real-time interpolation and better subjective quality. The ex- 
periment results are shown in the following. 
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[0041] Zero-order, bilinear and bicubic are well-known linear interpolation 
methods, and the PSNR and computational complexity of these algorithms are com- 
pared and shown in Table 1 and Table 4, respectively. Then, the interpolation algo- 
rithm of homogenous area is decided according to the above results. The PSNR of bi- 
linear and bicubic interpolation algorithms are similar, but the bicubic interpolation al- 
gorithm has greater computational complexity than bilinear interpolation algorithm. So, 
the bilinear interpolation algorithm is used in the first step of our proposed algorithm. 

[0042] Table 1 : Average PSNR values of the homogenous area using different 
algorithms for six images 





Zero-order 
(dB) 


Bilinear 
(dB) 


Bicubic 
(dB) 


PSNR 


26.61 


29.96 


30.05 



[0043] FIG. 9 shows the subjective quality for step edge test. In FIG. 9, the 
simple image is interpolated, and the result of NEDI algorithm has better performance 
than zero-order, bilinear and bicubic algorithms. The result of our proposed algorithm is 
similar to the NEDI algorithm. FIG. 10 and FIG. 11 show subjective qualities for the 
portions of Lena and Pepper images. In FIG. 10 and FIG. 11, the image edge is con- 
cerned, and the interpolated images have smoother on edge. Also and, the shoulders and 
the brim of a hat in FIG. 10 are observed, and pepper in FIG. 1 1 is observed. Table 2 
shows the objective quality of gray-level images for up sample from 256x 256 to 512x 
512 and from 128x 128 to 51 2x 512, and Table 3 shows the objective quality of color 
images for up sample from 256x 256 to 512x 512 and 128x 128 to 512x 512. The 
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proposed algorithm of the invention has better objective quality in the most of gray- 
level images. According to the above results, it is found that the NEDI algorithm has 
worse objective quality but has better subjective quality. The NEDI algorithm hides the 
distortion in homogenous area, and human eyes are not sensitive for homogenous area. 
5 So, our proposed algorithm has better objective quality than NEDI algorithm, and has 
similar subjective quality to NEDI algorithm. 



[0044] Table 2 PSNR(dB) comparison for gray level images 





256x256 to 512x512 


128x128 to 512x512 


Pep- 
per 


Milk- 
drop 


Tiffany 


Comtal 


Jet 


Lena 


Pepper 


Milk- 
drop 


Tif- 
fany 


Comtal 


Jet 


Lena 


Zero- 
order 


27.53 


29.85 


27.87 


25.39 


26.61 


29.01 


24.39 


26.36 


26.12 


22.01 


23.47 


25.87 


Bilinear 


32.19 


34.18 


29.90 


29.60 


30.79 


34.17 


27.52 


29.60 


27.77 


24.58 


25.91 


28.67 


Bicubic 


32.27 


34.31 


29.86 


29.73 


31.19 


34.68 


27.50 


29.66 


27.61 


24.50 


25.97 


28.75 


NEDI 


28.54 


29.20 


28.58 


27.06 


28.97 


30.10 


23.94 


24.94 


24.86 


22.21 


23.62 


24.98 


Pro- 
posed 


33.35 


35.34 


29.99 


29.95 


30.77 


33.95 


28.96 


31.16 


27.90 


24.77 


25.99 


28.56 
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[0045] Table 3 PSNR(dB) comparison for color images 





256x256 to 512x512 


128x128 to 512x512 


Pepper 


Jet 


Lena 


Pepper 


Jet 


Lena 


Zero- 
order 


26.14 


25.93 


28.04 


23.34 


23.19 


25.14 


Bilinear 


30.01 


29.35 


32.57 


26.25 


25.37 


27.77 


Bicubic 


29.95 


29.62 


32.86 


26.14 


25.38 


27.78 


NEDI 


27.06 


27.41 


29.11 


23.17 


23.00 


24.46 


Proposed 


30.84 


29.45 


32.32 


27.47 


25.49 


27.64 



[0046] Table 4 Comparison of computational complexity (Assume n pixels are 



interpolated) 





Add 


Sub 


Mul. 


Div 

■ 


Shift 


In- 
verse 


Zero-order 














Bilinear 


3n 


3n 


6n 








Bicubic 


27n 


45n 


135n 


9n 






NEDI 


Homoge- 
nous 


n 








n 




Edge 


4n 




1288n 






4n 


Pro- 
posed 


Homoge- 
nous 


n 








n 




Edge 


n 


4n 






n 
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[0047] The computational complexity of each algorithm is shown in Table 4. 
According to Table 4, the NEDI algorithm has the highest computational complexity on 
edge area. And, our proposed algorithm has lower computational complexity than most 
of other algorithms. 

[0048] In conclusions, the proposed algorithm at least successfully achieves two 
goals, which can have real-time interpolation and the subjective quality similar to edge 
direction interpolation. So, the proposed algorithm can be applied to videoconference, 
which transforms QCIF to CIF size or transforms CIF to 4CIF size. And, the quality of 
videoconference will be promoted. 

[0049] It will be apparent to those skilled in the art that various modifications 
and variations can be made to the structure of the present invention without departing 
from the scope or spirit of the invention. In view of the foregoing, it is intended that the 
present invention covers modifications and variations of this invention provided they 
fall within the scope of the following claims and their equivalents. 
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